Intelligent agents for managing data associated with three-dimensional objects

ABSTRACT

The techniques disclosed herein improve the efficiency of a system by providing intelligent agents for managing data associated with objects that are displayed within mixed-reality and virtual-reality collaboration environments and facilitating communication between the agents and computing devices associated with the collaboration environments. Individual agents are configured to collect, analyze, and store data associated with individual objects in a database associated with the agent. The agents are further configured to receive queries from the computing devices regarding an individual object. Agents can respond to queries by presenting relevant information collected by the agent with a view of the queried object of interest In addition, the data stored by the agents can be shared between different collaboration environments without requiring users to manually store and present a collection of content for each object. The intelligent agents can also persist across multiple collaboration environments to enhance user engagement and improve productivity.

PRIORITY INFORMATION

This application claims the benefit of and priority to U.S. patentapplication Ser. No. 17/202,324 filed Mar. 15, 2021 and entitled“INTELLIGENT AGENTS FOR MANAGING DATA ASSOCIATED WITH THREE-DIMENSIONALOBJECTS” which is a continuation of U.S. Pat. No. 10,970,547,application Ser. No. 16/213,867 filed Dec. 7, 2018 and entitled“INTELLIGENT AGENTS FOR MANAGING DATA ASSOCIATED WITH THREE-DIMENSIONALOBJECTS,” the entire contents of which are incorporated herein byreference.

BACKGROUND

Many productivity applications provide specialized tools for displayingand manipulating the contents of a file. Some productivity applicationsalso provide a shared workspace where multiple users can simultaneouslyview and edit the contents of a file from separate locations. Somesystems also allow multiple users to collaboratively edit content usingdevices that provide virtual reality (“VR”) and mixed reality (“MR”)environments.

Although current technologies can provide specialized functions forsharing and manipulating content, some existing applications do notprovide a satisfactory user experience when a user wishes to retrievedata or content from one or multiple shared workspaces to prepare avisual preview or synthesize information. For example, when groupconsensus is needed, users may have to manually prepare specializedcontent to allow other users to visualize each idea. A visual preview ofeach proposed idea can be helpful for the participants to gain anunderstanding or appreciation for each idea. However, the manual processof preparing each preview can be time consuming and inefficient when itcomes to computing resources, e.g., memory resources, processingresources, network resources, etc.

In addition, when users collaborate using one forum, such as a groupediting session using a Skype session, then switch to another forum,such as a private chat session, not all edited content can betransferred between the different types of sessions. This shortcomingcan lead to other inefficiencies with respect to computing resources asusers may be required to retrieve, transfer, or even re-create contenteach time they transition between different types of communicationsessions.

SUMMARY

The techniques disclosed herein improve the efficiency of a system byproviding intelligent agents for managing data associated withreal-world objects and virtual objects that are displayed withincollaborative environments and facilitating communication between theagents and a group of computing devices associated with thecollaborative environments. Individual agents are configured to collect,analyze, and store data associated with individual objects in a databaseassociated with the agent. The agents can identify real-world objectsand virtual objects discussed in a meeting, collect information abouteach object and receive user queries regarding an object. The agents arefurther configured to respond to user queries by presenting relevantinformation about the object. In addition, data stored by the agents canbe shared between different communication sessions without requiringusers to manually store and present a collection of content for eachobject. The intelligent agents can also persist through differentcommunication sessions to enhance user engagement and improveproductivity.

For example, if a first group of users is conducting a Skype meetingabout a car engine design, an intelligent agent can be instantiated foreach part of the engine. The intelligent agent can monitor all types ofuser activity during a multi-user communication session, e.g., polling,edits, text chats, and voice conversations. Data derived from themonitored activity can be displayed and stored in association with eachobject, e.g., each engine part. The agents are configured such that,when the Skype meeting ends, the agents and the stored data persistbeyond the communication session. Thus, the agents allow users to accessthe agents and the stored data when new communication channels, e.g.,private chat sessions or new meetings, are created.

Each agent can receive and respond to queries regarding individualobjects. The queries can be interpreted from communication datagenerated by various users such as emails, text messages, audio data,video calls and so forth. Individual agents can respond to queries byretrieving stored data from an associated database and presenting thedata to the users along with a view of the object in question. Datadefining the queries and stored data associated with each object canalso be stored persistently across communication sessions. Thus, when anobject, such as an engine part, is moved or deleted during a session,the associated queries and stored data are modified and persist acrossother communication sessions to indicate such changes.

The intelligent agents provide a number of features that improveexisting computers. For instance, computing resources such as processorcycles, memory, network bandwidth, and power, are used more efficientlyas users transition between different sessions. Data or content does notneed to be re-created for users to share and display content betweensessions. In addition, facilitating communication between computingdevices associated with various users and the intelligent agents enablesusers to make more efficient decisions and streamlines the userexperience by readily supplying relevant information for individualobjects managed by the agents. The techniques disclosed herein alsoimprove user interaction with various types of computing devices.Improvement of user interaction, or the reduction of a need for userinput, can mitigate inadvertent inputs, redundant inputs, and othertypes of user interactions that utilize computing resources. Othertechnical benefits not specifically mentioned herein can also berealized through implementations of the disclosed subject matter.

Those skilled in the art will also appreciate that aspects of thesubject matter described herein can be practiced on or in conjunctionwith other computer system configurations beyond those specificallydescribed herein, including multiprocessor systems, microprocessor-basedor programmable consumer electronics, AR, VR, and MR devices, video gamedevices, handheld computers, smartphones, smart televisions,self-driving vehicles, smart watches, e-readers, tablet computingdevices, special-purpose hardware devices, networked appliances, andother devices.

Features and technical benefits other than those explicitly describedabove will be apparent from a reading of the following DetailedDescription and a review of the associated drawings. This Summary isprovided to introduce a selection of concepts in a simplified form thatare further described below in the Detailed Description. This Summary isnot intended to identify key or essential features of the claimedsubject matter, nor is it intended to be used as an aid in determiningthe scope of the claimed subject matter. The term “techniques,” forinstance, may refer to system(s), method(s), computer-readableinstructions, module(s), algorithms, hardware logic, and/or operation(s)as permitted by the context described above and throughout the document.

BRIEF DESCRIPTION OF THE DRAWINGS

The Detailed Description is described with reference to the accompanyingfigures. In the figures, the left-most digit(s) of a reference numberidentifies the figure in which the reference number first appears. Thesame reference numbers in different figures indicate similar oridentical items. References made to individual items of a plurality ofitems can use a reference number with a letter of a sequence of lettersto refer to each individual item. Generic references to the items mayuse the specific reference number without the sequence of letters.

FIG. 1 illustrates a display of a collaborative environment generated bya computing device capturing an image of a real-world object.

FIG. 2 is a block diagram illustrating several intelligent agents thatare generated in association with virtual objects and real-worldobjects.

FIG. 3 illustrates a number of forms of communication that may beperformed by users interacting in a collaborative environment.

FIG. 4 is a block diagram illustrating several intelligent agents forcollecting FIG of a collaborative environment by the use of anapplication programming interface.

FIG. 5 is a block diagram illustrating a number of external resourcesthat may be utilized by the intelligent agents to collect supplementaldata associated with virtual objects and real-world objects.

FIG. 6 illustrates a display of a collaborative environment showing anumber of recommendations associated with virtual objects and real-worldobjects generated by the intelligent agents.

FIG. 7 illustrates a block diagram of a system having a management agentfor managing agent data between communication sessions.

FIG. 8 illustrates a display of a subsequent collaborative environmentinvolving a private chat that utilizes the intelligent agents created inthe first collaborative environment.

FIG. 9 illustrates a display of a subsequent collaborative environmentinvolving a Teams Session that utilizes the intelligent agents createdin the first collaborative environment.

FIG. 10 is a flow diagram illustrating aspects of a routine forcomputationally efficient intelligent agents for managing dataassociated with objects that are displayed within mixed-reality andvirtual-reality collaborative environments.

FIG. 11 is a computing system diagram showing aspects of an illustrativeoperating environment for the technologies disclosed herein.

FIG. 12 is a computing architecture diagram showing aspects of theconfiguration and operation of a computing device that can implementaspects of the technologies disclosed herein.

FIG. 13 is a computing device diagram showing aspects of theconfiguration and operation of a MR device that can implement aspects ofthe disclosed technologies, according to one embodiment disclosedherein.

DETAILED DESCRIPTION

FIG. 1 illustrates an example user interface 100 displaying aspects of acollaborative environment that may be utilized to facilitate aspects ofthe present disclosure. In this example, an image sensor 105, e.g., acamera, of the computing device 101 is used to capture an image of thereal-world object 103 within a real-world environment 112. In thisillustrative example, the real-world object 103 is an engine. Thecomputing device 101 can share an image of the real-world object 103with a number of individual users 102A-102D (“users 102”) that arecommunicating within the collaborative environment. The computing device101 can also coordinate with other computing devices to generate ashared display of virtual objects 104 that are rendered with a view of areal-world object 103.

The users 102 can create and edit the virtual objects 104 by the use ofvoice commands, movement gestures, or other types of interactions with awide range of input devices. In this illustrative example, a firstvirtual object 104A (a fan) and a second virtual object 104B (afastening device) are positioned at a specific location relative to thereal-world object 103. The users 102 can move and resize the virtualobjects 104. In addition, the users 102 can use one or more gestures,including a voice command, to associate the virtual objects 104 with oneor more real-world objects. In this illustrative example, the firstvirtual object 104A and the second virtual object 104B are associatedwith the real-world object 103, the engine. The association between eachobject can be based on a relative position between each object,graphical elements showing an association, text descriptions, or anyother suitable graphical representation. In this example, the firstvirtual object 104A and the second virtual object 104B are aligned withan axle of the engine to illustrate the association between the objects,e.g., how parts are to be assembled.

The users 102 can also communicate with one another within thecollaborative environment using a number of different mediums including,but not limited to: shared audio, video, and text communication. In thisexample, a first user, 102A, a second user 102B, a third user 102C, anda fourth user 102D are all in communication with one another, and avideo stream of each user 102 is rendered within the user interface 100.

The computing device 101 can identify both real-world objects andvirtual objects within the collaborative environment. For illustrativepurposes, general references to an “object” or “objects” refer to bothvirtual objects and real-world objects. The techniques disclosed hereincan use any number of object recognition technologies including, but notlimited to, edge detection, pattern detection, or shape detectionalgorithms, to identify an object. In this example, the computing device101 recognizes that the real-world object 103 is an engine by its shape,size, and other physical characteristics. One or more real-world objectscan also be identified by the use of sounds emitted by an object thatare detected by a microphone. Colors, shapes, patterns, surfacetextures, or other characteristics can be utilized to identify modelnumbers, manufacturers, parts, etc. Virtual objects can also beidentified by an analysis performed on model data defining each virtualobject. The shape, size or other characteristics can be interpreteddirectly from an analysis of the model data.

In response to identifying real-world objects and virtual objects thatare displayed within the collaborative environment, the computing device101 generates individual agents in association with each individualobject. Each agent 201 (shown in FIG. 2 ) can be an autonomous computingunit that is configured to collect and manage data associated with eachobject. Each agent is stored persistently, e.g., in a datastore thatmaintains all data associated with each agent 201 even after acommunication session associated with the collaboration environment isterminated. In some embodiments, each agent can be in the form of avirtual machine that can be stored on a single computing device andcommunicated between different computing devices.

In some embodiments, the computing device 101 can generate an agent forspecific objects of interest. The objects of interest can be selectedbased on physical properties of the objects or user activity. Forinstance, a particular object may be deemed an object of interest if theobject contains moving parts, has a size that exceeds a threshold, has apredetermined shape, etc. In another example, a particular object may bedeemed as an object of interest based on an analysis of user discussionsor preference data. Users 102 making comments about specific objects,such as the engine, can cause the computing device 101 to select theobject based on keywords or gestures associated with the object. In onespecific example, the engine shown in FIG. 1 can be deemed an object ofinterest if user communication includes a threshold number of keywordsor gestures associated with the engine. Other objects that are in directview of the camera 105 of the computing device 101, such as the walls ofa room, a light switch near the user, may not be deemed an object ofinterest if users have little interaction related to those objects.

In some embodiments, the computing device 101 can generate a score todetermine if a particular object is an object of interest. An object maybe selected as an object of interest when an associated score exceeds athreshold. A score may be generated based on a number factors. Forinstance, a score may be generated based on a number of times or afrequency of interactions a user has with a particular object. A scoremay also be based on movement of an object or other characteristics suchas size, shape, etc. The score can be used to rank objects. A rankingmay be utilized to select a group of objects having a predeterminedsize. For instance, a collaborative environment may have data defining amaximum number of objects for a particular communication session or aparticular computing device. In such embodiments, the highest rankedobjects, up to the maximum number of objects, can be selected as objectsof interest.

FIG. 2 illustrates an example implementation that includes a number ofagents 201 that are generated in association with identified objects. Inthis example, a first agent 201A is generated in association with thereal-world object 103, a second agent 201B is associated with the firstvirtual object 104A, and a third agent 201C is associated with thesecond virtual object 104B.

Each agent 201 manages a database 202 for storing data records (203-205)defining keywords, descriptions, parameters, or other data aboutindividual objects. In some embodiments, each database 202 can beconfigured to operate autonomously, e.g., each database 202 is stored ina data structure that can be independently communicated from onecomputing device to another computing device without impacting otherdatabases 202. In some embodiments, each database 202 is stored within adata structure, referred to herein as “agent data,” that also defines anassociated agent 201. As shown in FIG. 2 , the present example includesa first database 202A in communication with the first agent 201A, asecond database 202B in communication with the second agent 201B, and athird database 202C in communication with the third agent 201C.

As also shown in FIG. 2 , the computing device 101 can generate andmodify data records (203-205) associated with each object. The datarecords (203-205) may be generated based on a number of factors. In someembodiments, the data records (203-205) describing aspects of eachobject may be generated based on a shape, size, or other physicalcharacteristic of a real-world object or a virtual object. For instance,in the present example, a data record 203A describing a diameter of theengine driveshaft can be generated based on an analysis of an image ofthe engine, where the image can be used to measure parameters of aparticular component such as the driveshaft. Such data can be generatedby the analysis of depth map data and image data captured by thecomputing device. By the use of a measurement obtained from the depthmap data and image data, a computing device can determine geometries andshapes of real-world objects. Such measurements can be recorded in thedata records 203.

The computing device 101 can also analyze text or other insignia togenerate or modify data records (203-205) associated with each object.For example, a computing device may determine and record model numbers,product brands, or other related characteristics of an object. In thepresent example, a data record 203B describing the horsepower of theengine can be generated based on text inscribed on the engine or by thesize or shape of the engine. These examples are provided forillustrative purposes and are not to be construed as limiting. It can beappreciated that any type of keyword, description, or parameter of anobject can be generated by an analysis of a particular object.

The computing device 101 can also analyze model data defining virtualobjects and it can also be utilized to generate or modify data records(203-205) describing aspects of each object. For instance, in thepresent example, model data defining the first virtual object 104A canbe analyzed to determine that the object is made of a particularmaterial, such as steel. One or more records, such as record 204A, canbe generated by an associated agent, such as the second agent 201B, tostore such information. Also shown in FIG. 2 , another record 205Adescribing aspects, e.g., a size, of the second virtual object 104B isalso generated.

The agents 201 can also monitor a number of different types of useractivity during a multi-user communication session to generate or modifydata records (203-205). The monitored user activity can include any typeof user interaction with a computer or any type of communication, e.g.,polling, edits, text chats, and voice conversations. The techniquesdisclosed herein can monitor any type of interaction data defining auser input from an input device, a user sharing content, a user sendingor receiving streams over a communication session, or receiving content.As shown in FIG. 3 , the agents 201 can monitor a variety ofcommunication mediums including, but not limited to, emails, phonecalls, @mentions, video calls, text messages, audio data of acommunication session, etc. For instance, as shown in FIG. 3 , each ofthe agents may analyze an email string sent between the first user 102Aand the second user 102B to determine that the emails are related to thesecond virtual object 104B. Each of the other forms of communication,e.g., video broadcasts, @mention, or private calls, can also be analyzedand parsed to identify parameters, preferences, or other informationrelated to each object.

The respective agents 201 can collect and store activity data definingthe monitored activity in individual databases 202 associated with theirrespective objects. For example, as shown in FIG. 4 , based on themonitored communication between the users, each agent 201 stores thecollected information pertaining to each object. In some configurations,the collected information is received through an application programminginterface (API) in communication with each agent 201. Each instance ofcommunication, such as data from a phone call, an email, or an @mention,can be parsed and stored in a particular database 202. Data that isparsed from each instance of communication can be stored in a record(203-205).

The computing device 101 can associate activity data defining aparticular instance of user activity with an object if the particularinstance of user activity makes a reference to an object. For instance,keywords, phrases, images, audio data, or any other information thathave a defined threshold level of relevancy to a particular object cancause the computing device 101 to associate a particular instance ofuser activity with an object. Once an association with an object hasbeen made, the activity data defining the particular instance of useractivity can be stored in an associated database.

For example, the data of the broadcast can be analyzed to determine thatthe fan has specification requirements, e.g., that the fan requires aperformance of 200 cubic feet per minute (CFM). The analysis of suchcommunication can be stored in a database record, such as record 204B.The email can be analyzed to determine aspects of the fastening device,e.g., that the nut needs to have a particular strength, and the replyemail can be analyzed to determine parameters of the required strength,e.g., that the engine can produce a torque of 60 foot-pounds. Theanalysis of such communication can be stored in database records of theassociated objects. For example, record 203C of the first database 202Acan be generated to indicate the engine torque specification and record205B of the third database 202C can be generated to indicate a torquerequirement for the fan. Further, the @mention can be analyzed todetermine aspects of the first virtual object 104A, e.g., the fan. Inresponse, a record, such as record 204C, can be generated to indicatethat the fan needs to be made of carbon fiber.

Other forms of communication can be analyzed and parsed in a similarmanner and information pertaining to a particular object is stored in anassociated database. In the present example, a data record 203D, basedon the Skype call, indicates a need to increase the horsepower of theengine, and a data record 203E, based on the text, indicates arecommended brand-name for the engine.

Turning now to FIG. 5 , in some configurations, each agent 201 canretrieve supplemental data 501 from external resources 502. Thesupplemental data 501 can include any information pertaining to the datarecords associated with each object. For instance, queries can begenerated from existing data records (203-205), and each query can besent to various resources 502, such as, but not limited to, an inventorydatabase 502A, a company directory 502B, and a search engine 502C. Eachresource 502 can return supplemental data 501 in response to thequeries. In addition, each resource 502 can also push relevantsupplemental data 501 to each agent.

In one illustrative example, the first agent 201A can generate a querybased on the record stored in the first database 202A, e.g., records203A-203E. In one illustrative example, the first agent 201A maygenerate a query defining parameters of the real-world object, e.g., theshaft diameter, horsepower, brand-name, etc. In response to the query, aresource, such as the inventory database 502A may return supplementaldata 501 defining a new engine, e.g., Acme Model 1, that meets theperformance requirements indicated in the data records 203. Suchsupplemental data 501 can be stored in a new data record 203F in thefirst database 202A.

In another illustrative example, the second agent 201B may generate aquery defining aspects from the records of the first virtual object104A, e.g., that 200 CFM fan is required and that the model dataindicates a steel construction. In response to the query, a resource,such as the company directory 502B, can return supplemental data 501identifying individuals having expertise with such objects. Suchsupplemental data 501 can be stored in a new data record 204D in thesecond database 202B.

In yet another illustrative example, the third agent 201C may generate aquery based from the records associated with the second virtual object104B, e.g., that the fastening device as a particular size and requiresa particular strength. In response to the query, resource, such as thesearch engine 502C, can return supplemental data 501 that includes arecommendation to use a torque wrench. Such supplemental data 501 can bestored in a new data record 205C in the third database 202C. It can beappreciated that the supplemental data 501 can include data of anyformat, including three-dimensional model data, performance statistics,images, audio data, etc.

As shown in FIG. 6 , the computing device 101 can generaterecommendations for individual objects based on the stored information.The recommendations can suggest modifications to the objects, provideresources for obtaining or modifying the objects, and provide actionableinformation allowing users to reach a consensus regarding an object. Therecommendations can be in the form of (1) a modification of an extantvirtual object, (2) a new virtual object positioned over a real-worldobject, or (3) an annotation recommending a modification to a real-worldobject. Data defining the recommendations associated with each objectcan also be stored persistently within a data record (203-205).

FIG. 6 illustrates a number of example recommendations. For instance, arecommendation can be in the form of a new virtual object 601 that isdisplayed as a virtual partial overlay over the image of the real-worldobject. In this example, the new virtual object 601 is in the form ofanother engine, e.g., Acme Model 1, that meets the performancerequirements indicated in the data records 203. Such a recommendationmay be automatically rendered as a three-dimensional or two-dimensionalvirtual object that is provided as an overlay over real-world objects orother virtual objects. By providing an overlay over real-world objects,users can readily visualize a proposal without requiring users tomanually manipulate content.

The recommendations can also include computer-generated modifications ofa design based on the agent analysis of the communication data and thesupplemental data. As shown in FIG. 6 , a new virtual component 602,e.g., a washer, can be added to a schematic layout. The position andsize of the new virtual component 602 can be based on informationprovided in the supplemental data 501.

The agent recommendations (also referred to herein as “recommendations”)can also include graphical elements providing other contextualinformation 603 or instructions 604 related to an object. As shown inFIG. 6 , the graphical elements can be displayed in association with aparticular object. The contextual information 603 or the instruction 604can be generated from the supplemental data 501 or any other data recordassociated with an object. In the examples shown in FIG. 6 , somerecommendations indicate that a steel fan should be replaced with acarbon fiber fan, and identify vendors, etc.

The computing device 101 can prioritize and rank various recommendationsbased on the contents of the data records. In some configurations, thecomputing device 101 may store data defining a maximum number ofrecommendations that can be displayed. The computing device 101 maydisplay any number of recommendations up to a maximum number ofrecommendations. In some embodiments, the display of recommendations maybe arranged according to a defined priority of each recommendation,e.g., the recommendations may be ordered from a highest priority tolowest priority. Thus, the recommendations may also be prioritized basedon any type of contextual information. For instance, the recommendationsmay be prioritized and ranked based on votes or preferences of eachuser. In other examples, a priority for each recommendation may be basedon a number of comments made by various users regarding a particularobject, a number of interactions referencing a recommendation, or acombination of other types of user activity, such as a number of times auser looks at a particular object or recommendation.

Users can interact with the agent recommendations by selecting orotherwise interacting with the displayed recommendations. A user canselect a particular recommendation by providing a voice gesture or atouch gesture. In some configurations, a user interaction with aparticular recommendation can be interpreted as a vote or a preferencefor a particular recommendation. Communication data can also bemonitored to identify a vote or preference for a particularrecommendation. Supporting comments or votes can be tallied and when aparticular defined threshold of comments or votes is reached, thecomputing device 101 can determine a consensus for a particular group ofusers.

The computing device 101 can take a number of different actions when aconsensus is reached. For instance, when a threshold number of votes isreceived for a particular recommendation, the modification may bepermanently written to a database 202 associated with the particularobject. In another example, when a threshold number of votes is receivedfor a particular recommendation, the computing device 101 may generate asubsequent query based on the recommendation for additional supplementalinformation, which may in turn cause a generation of additionalrecommendations.

As summarized above, the intelligent agents 201 can persist throughdifferent communication sessions to enhance user engagement and improveproductivity. For example, if a first group of users is conducting aSkype meeting about a car engine design, an intelligent agent can beinstantiated for each part of the engine. The intelligent agent canmonitor all types of user activity during a multi-user communicationsession, e.g., polling, edits, text chats, and voice conversations. Dataderived from the monitored activity can be displayed and stored inassociation with each object, e.g., each engine part. The agents and theassociated databases are configured such that, when the Skype meetingends, the agents and the stored data persist beyond the communicationsession. Thus, the agents allow users to access the agents and thestored data when new communication channels, e.g., private chat sessionsor new meetings, are created.

Referring now to FIG. 7 , aspects of a computing device that enablespersistent storage of the agents and stored data is shown and describedbelow. In this illustrative example, a server 701 comprises memory 703storing agent data 704 that defines aspects of an individual agent 201and an associated database 202. In continuing the example describedabove, FIG. 7 shows a first agent data 704A that defines the first agent201A and the first database 202A, a second agent data 704B that definesa second agent 201B and the second database 202B, and a third agent data704C that defines a third agent 201C and a third database 202C.

In this example, a management agent 705 can receive agent data 704 fromany communication session such as a Skype meeting, a broadcast, or achat session, and the stored agent data 704 and memory 703. The memory703 can be configured to maintain the agent data 704 independent of thelifecycle of each communication session 1004. Thus, the management agent705 can deliver the agent data 704 to other communication sessions.

In the example shown in FIG. 7 , a first communication session 1004A,such as the collaboration environment illustrated in FIG. 1 , cangenerate agent data 704 defining individual agents 201 and associateddatabases 202. During the communication session, or at the conclusion ofthe communication session, agent data 704 can be communicated to theserver 701 executing the management agent 705. The management agent 705can store the agent data persistently in memory 703, which can be in theform of a database, persistent memory, or any other memory device thatallows the agent data to persist independent of the execution andtermination of any communication session.

Next, as other communication sessions are created, such as the secondcommunication session 1004B and the third communication session 1004C,the management agent 705 may provide relevant agent data 704 for eachsession. Agent data 704 defining a particular agent 201, may be selectedbased on one or more factors. For instance, if a new communicationsession includes content that has keywords describing a particularobject, agent data 704 that is related to that object may be deliveredto a computing device managing the communication session. In thisexample, second agent data 704B and third agent data 704C are deliveredto the second communication session 1004B, and the third agent data 704Cis delivered to the third communication session 1004C.

Referring now to FIG. 8 , an example user interface 800 for the secondcommunication session 1004B is shown and described below. In thisexample, the second communication session 1004B is in the form of aprivate chat session between two individuals, Sarah and Steve. In thisexample, it is a given that their conversation included a discussion onthe fan and the fastening device. In response to the agent detecting thecontext of the conversation, the objects defined in the relevant agentdata 704 are displayed in the user interface 800 of their communicationsession. This feature eliminates the need for users to gatherinformation and interact with computers to display that gatheredinformation within the session.

It can be appreciated that the second communication session 1004B can beprocessed as described above. Thus, the user activity of the privatechat session can be utilized to obtain additional supplemental data aswell as cause the generation of additional data records for each object.Also, additional recommendations may be generated and stored within newdata records associated with each object. As the users produce newvirtual objects or view new real-world objects, additional agents 201and associated databases 202 can be generated.

Referring now to FIG. 9 , an example user interface 900 for the thirdcommunication session 1004C is shown and described below. In thisexample, the third communication session 1004C is in the form of a TeamsSession between a large number of participants. In a Teams Session,multiple users are in communication via a chat window along with ashared video session 902. In this example, the participants start tochat about a particular object, the fastening device. In response todetecting keywords or other information related to an object defined inat least one database 202, such as the fastening device, a computingdevice managing the third communication session 1004C can retrieve theagent data 704C associated with the object. Based on the contents of theagent data 704C, the computer managing the third medication session1004C can render the relevant object and other information stored in theassociated data records. As shown in FIG. 9 , the second virtual object104B is rendered within the user interface 900 along with otherinformation, such as the related contextual information 603. Suchinformation can be automatically retrieved and automatically rendered bythe agent based on the context of the conversation. The second virtualobject 104B and the related contextual information 603 can be retrievedand displayed in response to a conversation object referencing, e.g.,asking about or mentioning, the virtual object.

The retrieval and display of contextually relevant information that isdisplayed in a usable format can improve the interaction between theusers and a computing device by reducing the need for manual operationsthat will be needed to carry out those operations. This feature canreduce inadvertent inputs and improve productivity for individuals whilealso reducing the use of computing resources required to carry out thosemanual operations.

It can be appreciated that the second communication session 1004B canalso be processed as described above. Thus, the user activity of theprivate chat session can be utilized to obtain additional supplementaldata as well as cause the generation of additional data records for eachobject. In addition, additional recommendations may be generated andstored within new data records associated with each object. As the usersproduce new virtual objects or view new real-world objects, additionalagents 201 and associated databases 202 can be generated.

In some configurations, machine learning techniques may be utilized toexamine the data records to generate recommendations. The term “machinelearning” may refer to one or more programs that learn from the data itreceives and analyzes. For example, a machine learning mechanism maybuild, modify or otherwise utilize a model that is created from exampleinputs and makes predictions or decisions using the model. The machinelearning mechanism may be used to improve the identification orgeneration of a recommendation based on requirements of an object oruser preferences. The model may be trained using supervised and/orunsupervised learning. For instance, over time as the machine learningmechanism receives more data, the recommendations displayed within acollaborative environment may change over time based on data defininguser activity.

Different machine learning mechanisms may be utilized. For example, aclassification mechanism may be utilized to determine an agentrecommendation based on requirements associated with an object and theavailability of other objects or information that meet thoserequirements. In another example, different classifications can indicatewhether users prefer or do not prefer a particular object or agentrecommendation. The classification mechanism may classify the displayelements into different categories that provide an indication of whetherthe display element should be displayed.

In other examples, a statistical mechanism may be utilized to determinewhether a particular agent recommendation should be displayed or whethera particular object is to be selected as an object of interest. Forexample, a linear regression mechanism may be used to generate a scorethat indicates a likelihood that an object is an object of interest.Linear regression may refer to a process for modeling the relationshipbetween one variable with one or more other variables. Different linearregression models might be used to calculate the score. For example, aleast squares approach might be utilized, a maximum-likelihoodestimation might be utilized, or another approach might be utilized.

FIG. 10 is a diagram illustrating aspects of a routine 1000 forcomputationally efficient management of data associated with objectsthat are displayed within mixed-reality and virtual-realitycollaboration environments. It should be understood by those of ordinaryskill in the art that the operations of the methods disclosed herein arenot necessarily presented in any particular order and that performanceof some or all of the operations in an alternative order(s) is possibleand is contemplated. The operations have been presented in thedemonstrated order for ease of description and illustration. Operationsmay be added, omitted, performed together, and/or performedsimultaneously, without departing from the scope of the appended claims.

It should also be understood that the illustrated methods can end at anytime and need not be performed in their entireties. Some or alloperations of the methods, and/or substantially equivalent operations,can be performed by execution of computer-readable instructions includedon a computer-storage media, as defined herein. The term“computer-readable instructions,” and variants thereof, as used in thedescription and claims, is used expansively herein to include routines,applications, application modules, program modules, programs,components, data structures, algorithms, and the like. Computer-readableinstructions can be implemented on various system configurations,including single-processor or multiprocessor systems, minicomputers,mainframe computers, personal computers, hand-held computing devices,microprocessor-based, programmable consumer electronics, combinationsthereof, and the like.

Thus, it should be appreciated that the logical operations describedherein are implemented (1) as a sequence of computer implemented acts orprogram modules running on a computing system such as those describedherein) and/or (2) as interconnected machine logic circuits or circuitmodules within the computing system. The implementation is a matter ofchoice dependent on the performance and other requirements of thecomputing system. Accordingly, the logical operations may be implementedin software, in firmware, in special purpose digital logic, and anycombination thereof.

Additionally, the operations illustrated in FIG. 10 and the otherFIGURES can be implemented in association with the example presentationUIs described above. For instance, the various device(s) and/ormodule(s) described herein can generate, transmit, receive, and/ordisplay data associated with content of a communication session (e.g.,live content, broadcasted event, recorded content, etc.) and/or apresentation UI that includes renderings of one or more participants ofremote computing devices, avatars, channels, chat sessions, videostreams, images, virtual objects, and/or applications associated with acommunication session.

The routine 1000 begins at operation 1002, where the computing device101 receives sensor data that defines a 3D representation of areal-world environment. The sensor data can be captured by a depth mapsensor, e.g., a depth map camera. In addition, the sensor data can becaptured by an image sensor, e.g. a camera, where the depth map sensorand the image sensor can be part of the same component or in separatecomponents. The sensor data comprises depth map data defining athree-dimensional model of a real-world environment and an image of thereal-world environment. For instance, a real-world environment mayinclude the walls of a room and a particular object within the room,such as the real-world object shown in FIG. 1 . The sensor data candefine physical properties of an object or a plurality of real-worldobjects within the real-world environment. The sensor data alsoindicates a geographic position of one or more objects within anenvironment. Thus, measurements of an object or measurements of theenvironment can be made by an analysis of the sensor data. One or moreobjects defined in the sensor data are shared with the number of usersparticipating in a collaborative environment. The collaborativeenvironment can include a communication session that allows users tosend, receive and view aspects of the sensor data rendered on a displaydevice.

The routine then proceeds to operation 1004, where the computing device101 receives model data defining one or more virtual objects to bedisplayed within a view of the collaborative environment. The model datacan define specific positions where the virtual objects are to be placedwithin a user interface of the collaborative environment.

At operation 1006, the computing device can identify virtual objects andreal-world objects of interest. As described herein, objects can bedeemed as objects of interest based on a number of factors, includingbut not limited to, a threshold level of user interaction with aparticular object, a threshold level of communication regarding aparticular object, a threshold level of movement of a particular object,etc.

In some embodiments, an object may be deemed as an object of interest bydetermining a level of interaction of input signals received from inputdevices with respect to at least one real-world object of the pluralityof real-world objects. This may include a number of inputs for editing aparticular object, a number of comments made about a particular object,the person's attention being drawn to a particular item, such as anamount of time someone spends looking at a particular item, etc. Thiscan be determined by tracking eye movement of a particular user andnoting how much time a person spends looking at an object over a periodof time. A system can determine when a level of interaction exceeds adefined threshold. In response to determining that the level ofinteraction exceeds the threshold, a system can determine that at leastone object is an object of interest. Such techniques can also apply toaspects of an object, such as the size of an object, the amount ofmovement of an object, a color of an object, a temperature of an object,a texture of an object, etc. A level of movement can mean a velocity, adistance, a rotational speed, or any other type of movement. When themovement exceeds a threshold, a system can deem a particular object tobe an object of interest.

A system can deem a particular object as an object of interest inresponse to detecting a threshold level of change with respect to atleast one of a temperature, a color, or a physical property of thatparticular object. The physical property changes can be, for instance, ashape, e.g., that an object is melting, from a liquid to a solid, from asolid to a liquid, etc.

A system can deem a particular object as an object of interest inresponse to determining that a physical property of the particularobject matches one or more predetermined properties. For instance, asystem may have a preference file indicating predetermined properties,such as a visual profile of an object, such as an engine or a computer.If the image data or the depth map data indicate that a particularreal-world object has physical properties, e.g., a dimension, color, orsize, that match, at least within a threshold difference, physicalproperties described in a preference file, that particular real-worldobject may be selected as an object of interest. This feature may allowa system to select, for instance, all the computers in a room, or allthe engines in a view. This can eliminate unwanted objects from beingidentified such as walls, furniture, or people. This technique also canallow a computing device to be more efficient by only creating agentsand databases for particular objects in a meeting. Thus, agents willonly be generated for items that are considered to be salient in ameeting.

Next, at operation 1008, the computing device can generate one or moreagents and associated databases associated with the virtual objects ofinterest. For illustrative purposes, the generation of an agent isreferred to herein as the generation of data defining an agent instance,wherein an agent instance is associated with at least one object. Datadefining the agent instance is stored persistently for access bymultiple communication sessions. As described herein, individual agentsare instantiated for identified objects. The individual agents areconfigured to analyze aspects of the object to generate descriptions,keywords, or other information regarding the object.

At operation 1010, the computing device monitors user activity toidentify information associated with each object. Keyword descriptionsor other parameters associated with an object are collected from varioussources such as input devices, search engines, inventory databases,committee directories, etc.

At operation 1012, the computing device can update individual databaseswith the identified information. As described herein, any informationcollected in association with an object can be stored by an agent andstored within a database associated with the object.

At operation 1014, the computing device can generate one or morerecommendations associated with individual objects. The recommendationscan include a recommendation for modifying an object. Therecommendations can be in the form of (1) a modification of an extantvirtual object, (2) a new virtual object positioned over a real-worldobject, or (3) an annotation recommending a modification to a real-worldobject. The recommendations can also include the display of statisticaldata, performance parameters, requirements, and other contextual data.

Data defining the recommendations associated with each object can alsobe stored persistently across communication sessions. Thus, when anobject, such as an engine part, is moved or deleted during a session,the recommendation is modified in other communication sessions toindicate such changes. For instance, if the first communication sessiondescribed above is concurrently running with the second communicationsession, modifications to a particular object within the firstcommunication session will update the record data and in turn, update adisplay of the particular object or recommendation of the object withinthe second communication session.

In some configurations, operation 1014 can include generating a virtualdisplay of one or more recommendations associated with a rendering ofthe individual virtual objects and a view of at least one real-worldobject. The recommendations can be in at least one of the followingformats as follows. In one example, a recommendation can includemodifications to at least one virtual object comprising at least one newparameter. For example, a recommendation may actually modify the secondvirtual object, such as the fastening device, to have a different size,a different color, a different shape, a different texture, etc. Inanother example, the recommendation can include a new virtual objectpositioned over a real-world object. This is illustrated in theabove-described example where a virtual model of a new engine isdisplayed over the real-world engine. In yet another example, arecommendation can include an annotation recommending a modification toat least one real-world object or a virtual object. As shown in theexample of FIG. 6 , a number of annotations can provide any informationfor an associated object, such as a new size, a new color, a differentshape, or any other type of modification. In addition, an annotation canprovide vendor names, contact names, or any other contextual data thatmay be useful for obtaining a particular item. Other information such asvoting results, consensus decisions or any other decision-making datamay be displayed in association with a particular object.

At operation 1016, the computing device can store and process therecommendations for machine learning purposes. As described herein, anycollected information such as the communication data or the supplementaldata, or any generated information such as a recommendation can beprovided as input to a machine learning algorithm to improve thegeneration of future recommendations.

It should be appreciated that the above-described subject matter may beimplemented as a computer-controlled apparatus, a computer process, acomputing system, or as an article of manufacture such as acomputer-readable storage medium. The operations of the example methodsare illustrated in individual blocks and summarized with reference tothose blocks. The methods are illustrated as logical flows of blocks,each block of which can represent one or more operations that can beimplemented in hardware, software, or a combination thereof. In thecontext of software, the operations represent computer-executableinstructions stored on one or more computer-readable media that, whenexecuted by one or more processors, enable the one or more processors toperform the recited operations.

Generally, computer-executable instructions include routines, programs,objects, modules, components, data structures, and the like that performparticular functions or implement particular abstract data types. Theorder in which the operations are described is not intended to beconstrued as a limitation, and any number of the described operationscan be executed in any order, combined in any order, subdivided intomultiple sub-operations, and/or executed in parallel to implement thedescribed processes. The described processes can be performed byresources associated with one or more device(s) such as one or moreinternal or external CPUs or GPUs, and/or one or more pieces of hardwarelogic such as field-programmable gate arrays (“FPGAs”), digital signalprocessors (“DSPs”), or other types of accelerators.

All of the methods and processes described above may be embodied in, andfully automated via, software code modules executed by one or moregeneral purpose computers or processors. The code modules may be storedin any type of computer-readable storage medium or other computerstorage device, such as those described below. Some or all of themethods may alternatively be embodied in specialized computer hardware,such as that described below.

Any routine descriptions, elements or blocks in the flow diagramsdescribed herein and/or depicted in the attached figures should beunderstood as potentially representing modules, segments, or portions ofcode that include one or more executable instructions for implementingspecific logical functions or elements in the routine. Alternateimplementations are included within the scope of the examples describedherein in which elements or functions may be deleted, or executed out oforder from that shown or discussed, including substantiallysynchronously or in reverse order, depending on the functionalityinvolved as would be understood by those skilled in the art.

FIG. 11 is a diagram illustrating an example environment 1100 in which asystem 1102 can implement the techniques disclosed herein. In someimplementations, a system 1102 may function to collect, analyze, sharedata defining one or more objects that are displayed to users of acommunication session 1004.

As illustrated, the communication session 1104 may be implementedbetween a number of client computing devices 1106(1) through 1106(N)(where N is a number having a value of two or greater) that areassociated with the system 1102 or are part of the system 1102. Theclient computing devices 1106(1) through 1106(N) enable users, alsoreferred to as individuals, to participate in the communication session1104. For instance, the first client computing device 1106(1) may be thecomputing device 101 of FIG. 1 or the computing device 1300 of FIG. 13 .

In this example, the communication session 1104 is hosted, over one ormore network(s) 1108, by the system 1102. That is, the system 1102 canprovide a service that enables users of the client computing devices1106(1) through 1106(N) to participate in the communication session 1104(e.g., via a live viewing and/or a recorded viewing). Consequently, a“participant” to the communication session 1104 can comprise a userand/or a client computing device (e.g., multiple users may be in a roomparticipating in a communication session via the use of a single clientcomputing device), each of which can communicate with otherparticipants. As an alternative, the communication session 1104 can behosted by one of the client computing devices 1106(1) through 1106(N)utilizing peer-to-peer technologies. The system 1102 can also host chatconversations and other team collaboration functionality (e.g., as partof an application suite).

In some implementations, such chat conversations and other teamcollaboration functionality are considered external communicationsessions distinct from the communication session 1104. A computerizedagent to collect participant data in the communication session 1104 maybe able to link to such external communication sessions. Therefore, thecomputerized agent may receive information, such as date, time, sessionparticulars, and the like, that enables connectivity to such externalcommunication sessions. In one example, a chat conversation can beconducted in accordance with the communication session 1104.Additionally, the system 1102 may host the communication session 1104,which includes at least a plurality of participants co-located at ameeting location, such as a meeting room or auditorium, or located indisparate locations.

In examples described herein, client computing devices 1106(1) through1106(N) participating in the communication session 1104 are configuredto receive and render for display, on a user interface of a displayscreen, communication data. The communication data can comprise acollection of various instances, or streams, of live content and/orrecorded content. The collection of various instances, or streams, oflive content and/or recorded content may be provided by one or morecameras, such as video cameras. For example, an individual stream oflive or recorded content can comprise media data associated with a videofeed provided by a video camera (e.g., audio and visual data thatcapture the appearance and speech of a user participating in thecommunication session). In some implementations, the video feeds maycomprise such audio and visual data, one or more still images, and/orone or more avatars. The one or more still images may also comprise oneor more avatars.

Another example of an individual stream of live or recorded content cancomprise media data that includes an avatar of a user participating inthe communication session along with audio data that captures the speechof the user. Yet another example of an individual stream of live orrecorded content can comprise media data that includes a file displayedon a display screen along with audio data that captures the speech of auser. Accordingly, the various streams of live or recorded contentwithin the communication data enable a remote meeting to be facilitatedbetween a group of people and the sharing of content within the group ofpeople. In some implementations, the various streams of live or recordedcontent within the communication data may originate from a plurality ofco-located video cameras, positioned in a space, such as a room, torecord or stream live a presentation that includes one or moreindividuals presenting and one or more individuals consuming presentedcontent.

A participant or attendee can view content of the communication session1104 live as activity occurs, or alternatively, via a recording at alater time after the activity occurs. In examples described herein,client computing devices 1106(1) through 1106(N) participating in thecommunication session 1104 are configured to receive and render fordisplay, on a user interface of a display screen, communication data.The communication data can comprise a collection of various instances,or streams, of live and/or recorded content. For example, an individualstream of content can comprise media data associated with a video feed(e.g., audio and visual data that capture the appearance and speech of auser participating in the communication session). Another example of anindividual stream of content can comprise media data that includes anavatar of a user participating in the conference session along withaudio data that captures the speech of the user. Yet another example ofan individual stream of content can comprise media data that includes acontent item displayed on a display screen and/or audio data thatcaptures the speech of a user. Accordingly, the various streams ofcontent within the communication data enable a meeting or a broadcastpresentation to be facilitated amongst a group of people dispersedacross remote locations.

A participant or attendee to a communication session is a person that isin range of a camera, or other image and/or audio capture device suchthat actions and/or sounds of the person which are produced while theperson is viewing and/or listening to the content being shared via thecommunication session can be captured (e.g., recorded). For instance, aparticipant may be sitting in a crowd viewing the shared content live ata broadcast location where a stage presentation occurs. Or a participantmay be sitting in an office conference room viewing the shared contentof a communication session with other colleagues via a display screen.Even further, a participant may be sitting or standing in front of apersonal device (e.g., tablet, smartphone, computer, etc.) viewing theshared content of a communication session alone in their office or athome.

The system 1102 includes device(s) 1110. The device(s) 1110 and/or othercomponents of the system 1102 can include distributed computingresources that communicate with one another and/or with the clientcomputing devices 1106(1) through 1106(N) via the one or more network(s)1108. In some examples, the system 1102 may be an independent systemthat is tasked with managing aspects of one or more communicationsessions such as communication session 1104. As an example, the system1102 may be managed by entities such as SLACK, WEBEX, GOTOMEETING,GOOGLE HANGOUTS, etc.

Network(s) 1108 may include, for example, public networks such as theInternet, private networks such as an institutional and/or personalintranet, or some combination of private and public networks. Network(s)1108 may also include any type of wired and/or wireless network,including but not limited to local area networks (“LANs”), wide areanetworks (“WANs”), satellite networks, cable networks, Wi-Fi networks,WiMax networks, mobile communications networks (e.g., 3G, 4G, and soforth) or any combination thereof. Network(s) 1108 may utilizecommunications protocols, including packet-based and/or datagram-basedprotocols such as Internet protocol (“IP”), transmission controlprotocol (“TCP”), user datagram protocol (“UDP”), or other types ofprotocols. Moreover, network(s) 1108 may also include a number ofdevices that facilitate network communications and/or form a hardwarebasis for the networks, such as switches, routers, gateways, accesspoints, firewalls, base stations, repeaters, backbone devices, and thelike.

In some examples, network(s) 1108 may further include devices thatenable connection to a wireless network, such as a wireless access point(“WAP”). Examples support connectivity through WAPs that send andreceive data over various electromagnetic frequencies (e.g., radiofrequencies), including WAPs that support Institute of Electrical andElectronics Engineers (“IEEE”) 802.11 standards (e.g., 802.11g, 802.11n,802.11ac and so forth), and other standards.

In various examples, device(s) 1110 may include one or more computingdevices that operate in a cluster or other grouped configuration toshare resources, balance load, increase performance, provide fail-oversupport or redundancy, or for other purposes. For instance, device(s)1110 may belong to a variety of classes of devices such as traditionalserver-type devices, desktop computer-type devices, and/or mobile-typedevices. Thus, although illustrated as a single type of device or aserver-type device, device(s) 1110 may include a diverse variety ofdevice types and are not limited to a particular type of device.Device(s) 1110 may represent, but are not limited to, server computers,desktop computers, web-server computers, personal computers, mobilecomputers, laptop computers, tablet computers, or any other sort ofcomputing device.

A client computing device (e.g., one of client computing device(s)1106(1) through 1106(N)) may belong to a variety of classes of devices,which may be the same as, or different from, device(s) 1110, such astraditional client-type devices, desktop computer-type devices,mobile-type devices, special purpose-type devices, embedded-typedevices, and/or wearable-type devices. Thus, a client computing devicecan include, but is not limited to, a desktop computer, a game consoleand/or a gaming device, a tablet computer, a personal data assistant(“PDA”), a mobile phone/tablet hybrid, a laptop computer, atelecommunication device, a computer navigation type client computingdevice such as a satellite-based navigation system including a globalpositioning system (“GPS”) device, a wearable device, a virtual reality(“VR”) device, an augmented reality (“AR”) device, an implantedcomputing device, an automotive computer, a network-enabled television,a thin client, a terminal, an Internet of Things (“IoT”) device, a workstation, a media player, a personal video recorder (“PVR”), a set-topbox, a camera, an integrated component (e.g., a peripheral device) forinclusion in a computing device, an appliance, or any other sort ofcomputing device. Moreover, the client computing device may include acombination of the earlier listed examples of the client computingdevice such as, for example, desktop computer-type devices or amobile-type device in combination with a wearable device, etc.

Client computing device(s) 1106(1) through 1106(N) of the variousclasses and device types can represent any type of computing devicehaving one or more data processing unit(s) 1112 operably connected tocomputer-readable media 1184 such as via a bus 1116, which in someinstances can include one or more of a system bus, a data bus, anaddress bus, a PCI bus, a Mini-PCI bus, and any variety of local,peripheral, and/or independent buses.

Executable instructions stored on computer-readable media 1194 mayinclude, for example, an operating system 1119, a client module 1120, aprofile module 1122, and other modules, programs, or applications thatare loadable and executable by data processing units(s) 1192.

Client computing device(s) 1106(1) through 1106(N) may also include oneor more interface(s) 1124 to enable communications between clientcomputing device(s) 1106(1) through 1106(N) and other networked devices,such as device(s) 1110, over network(s) 1108. Such network interface(s)1124 may include one or more network interface controllers (NICs) orother types of transceiver devices to send and receive communicationsand/or data over a network. Moreover, client computing device(s) 1106(1)through 1106(N) can include input/output (“I/O”) interfaces (devices)1126 that enable communications with input/output devices such as userinput devices including peripheral input devices (e.g., a gamecontroller, a keyboard, a mouse, a pen, a voice input device such as amicrophone, a video camera for obtaining and providing video feedsand/or still images, a touch input device, a gestural input device, andthe like) and/or output devices including peripheral output devices(e.g., a display, a printer, audio speakers, a haptic output device, andthe like). FIG. 11 illustrates that client computing device 1106(1) isin some way connected to a display device (e.g., a display screen1129(1)), which can display a UI according to the techniques describedherein.

In the example environment 1100 of FIG. 11 , client computing devices1106(1) through 1106(N) may use their respective client modules 1120 toconnect with one another and/or other external device(s) in order toparticipate in the communication session 1104, or in order to contributeactivity to a collaboration environment. For instance, a first user mayutilize a client computing device 1106(1) to communicate with a seconduser of another client computing device 1106(2). When executing clientmodules 1120, the users may share data, which may cause the clientcomputing device 1106(1) to connect to the system 1102 and/or the otherclient computing devices 1106(2) through 1106(N) over the network(s)1108.

The client computing device(s) 1106(1) through 1106(N) may use theirrespective profile modules 1122 to generate participant profiles (notshown in FIG. 11 ) and provide the participant profiles to other clientcomputing devices and/or to the device(s) 1110 of the system 1102. Aparticipant profile may include one or more of an identity of a user ora group of users (e.g., a name, a unique identifier (“ID”), etc.), userdata such as personal data, machine data such as location (e.g., an IPaddress, a room in a building, etc.) and technical capabilities, etc.Participant profiles may be utilized to register participants forcommunication sessions.

As shown in FIG. 11 , the device(s) 1110 of the system 1102 include aserver module 1130 and an output module 1132. In this example, theserver module 1130 is configured to receive, from individual clientcomputing devices such as client computing devices 1106(1) through1106(N), media streams 1134(1) through 1134(N). As described above,media streams can comprise a video feed (e.g., audio and visual dataassociated with a user), audio data which is to be output with apresentation of an avatar of a user (e.g., an audio only experience inwhich video data of the user is not transmitted), text data (e.g., textmessages), file data and/or screen sharing data (e.g., a document, aslide deck, an image, a video displayed on a display screen, etc.), andso forth. Thus, the server module 1130 is configured to receive acollection of various media streams 1134(1) through 1134(N) during alive viewing of the communication session 1104 (the collection beingreferred to herein as “media data 1134”). In some scenarios, not all ofthe client computing devices that participate in the communicationsession 1104 provide a media stream. For example, a client computingdevice may only be a consuming, or a “listening”, device such that itonly receives content associated with the communication session 1104 butdoes not provide any content to the communication session 1104.

In various examples, the server module 1130 can select aspects of themedia streams 1134 that are to be shared with individual ones of theparticipating client computing devices 1106(1) through 1106(N).Consequently, the server module 1130 may be configured to generatesession data 1136 based on the streams 1134 and/or pass the session data1136 to the output module 1132. Then, the output module 1132 maycommunicate communication data 1138 to the client computing devices(e.g., client computing devices 1106(1) through 1106(3) participating ina live viewing of the communication session). The communication data1138 may include video, audio, and/or other content data, provided bythe output module 1132 based on content 1150 associated with the outputmodule 1132 and based on received session data 1136.

As shown, the output module 1132 transmits communication data 1139(1) toclient computing device 1106(1), and transmits communication data1139(2) to client computing device 1106(2), and transmits communicationdata 1139(3) to client computing device 1106(3), etc. The communicationdata 1139 transmitted to the client computing devices can be the same orcan be different (e.g., positioning of streams of content within a userinterface may vary from one device to the next).

In various implementations, the device(s) 1110 and/or the client module1120 can include UI presentation module 1140. The UI presentation module1140 may be configured to analyze communication data 1139 that is fordelivery to one or more of the client computing devices 1106.Specifically, the UI presentation module 1140, at the device(s) 1110and/or the client computing device 1106, may analyze communication data1139 to determine an appropriate manner for displaying video, image,and/or content on the display screen 1129 of an associated clientcomputing device 1106. In some implementations, the UI presentationmodule 1140 may provide video, image, and/or content to a presentationUI 1146 rendered on the display screen 1129 of the associated clientcomputing device 1106. The presentation UI 1146 may be caused to berendered on the display screen 1129 by the UI presentation module 1140.The presentation UI 1146 may include the video, image, and/or contentanalyzed by the UI presentation module 1140.

In some implementations, the presentation UI 1146 may include aplurality of sections or grids that may render or comprise video, image,and/or content for display on the display screen 1129. For example, afirst section of the presentation UI 1146 may include a video feed of apresenter or individual, a second section of the presentation UI 1146may include a video feed of an individual consuming meeting informationprovided by the presenter or individual. The UI presentation module 1140may populate the first and second sections of the presentation UI 1146in a manner that properly imitates an environment experience that thepresenter and the individual may be sharing.

In some implementations, the UI presentation module 1140 may enlarge orprovide a zoomed view of the individual represented by the video feed inorder to highlight a reaction, such as a facial feature, the individualhad to the presenter. In some implementations, the presentation UI 1146may include a video feed of a plurality of participants associated witha meeting, such as a general communication session. In otherimplementations, the presentation UI 1146 may be associated with achannel, such as a chat channel, enterprise teams channel, or the like.Therefore, the presentation UI 1146 may be associated with an externalcommunication session that is different than the general communicationsession.

FIG. 12 illustrates a diagram that shows example components of anexample device 1200 (also referred to herein as a “computing device”)configured to generate data for some of the user interfaces disclosedherein. The device 1200 may generate data that may include one or moresections that may render or comprise video, images, virtual objects 116,and/or content for display on the display screen 1129. The device 1200may represent one of the device(s) described herein. Additionally, oralternatively, the device 1200 may represent one of the client computingdevices 1106.

As illustrated, the device 1200 includes one or more data processingunit(s) 1202, computer-readable media 1204, and communicationinterface(s) 1206. The components of the device 1200 are operativelyconnected, for example, via a bus 1208, which may include one or more ofa system bus, a data bus, an address bus, a PCI bus, a Mini-PCI bus, andany variety of local, peripheral, and/or independent buses.

As utilized herein, data processing unit(s), such as the data processingunit(s) 1202 and/or data processing unit(s)1192, may represent, forexample, a CPU-type data processing unit, a GPU-type data processingunit, a field-programmable gate array (“FPGA”), another class of DSP, orother hardware logic components that may, in some instances, be drivenby a CPU. For example, and without limitation, illustrative types ofhardware logic components that may be utilized includeApplication-Specific Integrated Circuits (“ASICs”), Application-SpecificStandard Products (“ASSPs”), System-on-a-Chip Systems (“SOCs”), ComplexProgrammable Logic Devices (“CPLDs”), etc.

As utilized herein, computer-readable media, such as computer-readablemedia 1204 and computer-readable media 1194, may store instructionsexecutable by the data processing unit(s). The computer-readable mediamay also store instructions executable by external data processing unitssuch as by an external CPU, an external GPU, and/or executable by anexternal accelerator, such as an FPGA type accelerator, a DSP typeaccelerator, or any other internal or external accelerator. In variousexamples, at least one CPU, GPU, and/or accelerator is incorporated in acomputing device, while in some examples one or more of a CPU, GPU,and/or accelerator is external to a computing device.

Computer-readable media, which might also be referred to herein as acomputer-readable medium, may include computer storage media and/orcommunication media. Computer storage media may include one or more ofvolatile memory, nonvolatile memory, and/or other persistent and/orauxiliary computer storage media, removable and non-removable computerstorage media implemented in any method or technology for storage ofinformation such as computer-readable instructions, data structures,program modules, or other data. Thus, computer storage media includestangible and/or physical forms of media included in a device and/orhardware component that is part of a device or external to a device,including but not limited to random access memory (“RAM”), staticrandom-access memory (“SRAM”), dynamic random-access memory (“DRAM”),phase change memory (“PCM”), read-only memory (“ROM”), erasableprogrammable read-only memory (“EPROM”), electrically erasableprogrammable read-only memory (“EEPROM”), flash memory, compact discread-only memory (“CD-ROM”), digital versatile disks (“DVDs”), opticalcards or other optical storage media, magnetic cassettes, magnetic tape,magnetic disk storage, magnetic cards or other magnetic storage devicesor media, solid-state memory devices, storage arrays, network attachedstorage, storage area networks, hosted computer storage or any otherstorage memory, storage device, and/or storage medium that can be usedto store and maintain information for access by a computing device.

In contrast to computer storage media, communication media may embodycomputer-readable instructions, data structures, program modules, orother data in a modulated data signal, such as a carrier wave, or othertransmission mechanism. As defined herein, computer storage media doesnot include communication media. That is, computer storage media doesnot include communications media consisting solely of a modulated datasignal, a carrier wave, or a propagated signal, per se.

Communication interface(s) 1206 may represent, for example, networkinterface controllers (“NICs”) or other types of transceiver devices tosend and receive communications over a network. Furthermore, thecommunication interface(s) 1206 may include one or more video camerasand/or audio devices 1222 to enable generation of video feeds and/orstill images, and so forth.

In the illustrated example, computer-readable media 1204 includes a datastore 1208. In some examples, the data store 1208 includes data storagesuch as a database, data warehouse, or other type of structured orunstructured data storage. In some examples, the data store 1208includes a corpus and/or a relational database with one or more tables,indices, stored procedures, and so forth to enable data access includingone or more of hypertext markup language (“HTML”) tables, resourcedescription framework (“RDF”) tables, web ontology language (“OWL”)tables, and/or extensible markup language (“XML”) tables, for example.

The data store 1208 may store data for the operations of processes,applications, components, and/or modules stored in computer-readablemedia 1204 and/or executed by data processing unit(s) 1202 and/oraccelerator(s). For instance, in some examples, the data store 1208 maystore session data 1210 (e.g., session data 1136), profile data 1212(e.g., associated with a participant profile), and/or other data. Thesession data 1210 can include a total number of participants (e.g.,users and/or client computing devices) in a communication session,activity that occurs in the communication session, a list of invitees tothe communication session, and/or other data related to when and how thecommunication session is conducted or hosted. The data store 1208 mayalso include content data 1214, such as the content that includes video,audio, or other content for rendering and display on one or more of thedisplay screens 1129.

Alternately, some or all of the above-referenced data can be stored onseparate memories 1216 on board one or more data processing unit(s) 1202such as a memory on board a CPU-type processor, a GPU-type processor, anFPGA-type accelerator, a DSP-type accelerator, and/or anotheraccelerator. In this example, the computer-readable media 1204 alsoincludes an operating system 1218 and application programminginterface(s) 1210 (APIs) configured to expose the functionality and thedata of the device 1200 to other devices. Additionally, thecomputer-readable media 1204 includes one or more modules such as theserver module 1230, the output module 1232, and the GUI presentationmodule 1240, although the number of illustrated modules is just anexample, and the number may vary higher or lower. That is, functionalitydescribed herein in association with the illustrated modules may beperformed by a fewer number of modules or a larger number of modules onone device or spread across multiple devices.

FIG. 13 is a computing device diagram showing aspects of theconfiguration and operation of a computing device 1300 that canimplement aspects of the systems disclosed herein. The computing device1300 shows details of the computing device 101 shown in FIG. 1 . Thecomputing device 1300 can provide augmented reality (“AR”) environmentsor virtual reality (“VR”) environments. Generally described, ARenvironments superimpose computer-generated (“CG”) images over a user'sview of a real-world environment. For example, a computing device 1300can generate composite views to enable a user to visually perceive acomputer-generated image superimposed over a rendering of a real-worldenvironment 112, wherein the rendering of the real-world environment 112is created by a camera 105 directed to the real-world environment, suchas a room. In some embodiments, a computing device 1300 can generatecomposite views to enable a user to visually perceive acomputer-generated image superimposed over a direct view of a real-worldenvironment 112. Thus, the computing device 1300 may have a prism orother optical device that allows a user to see through the opticaldevice to see a direct view of a real-world object or a real-worldenvironment, and at the same time, a computer-generated imagesuperimposed over that view of a real-world object. An AR environmentcan also be referred to herein as a mixed reality (“MR”) environment. AnMR device can provide both AR and VR environments. A VR environmentincludes computer-generated images of a virtual environment and virtualobjects. MR and AR environments can utilize depth map sensors todetermine a distance between the device and a real-world object. Thisallows the computer to scale and position a computer-generated graphicover a real-world object in a realistic manner.

In the example shown in FIG. 13 , an optical system 1302 includes anillumination engine 1304 to generate electromagnetic (“EM”) radiationthat includes both a first bandwidth for generating CG images and asecond bandwidth for tracking physical objects. The first bandwidth mayinclude some or all of the visible-light portion of the EM spectrumwhereas the second bandwidth may include any portion of the EM spectrumthat is suitable to deploy a desired tracking protocol. In this example,the optical system 1302 further includes an optical assembly 1306 thatis positioned to receive the EM radiation from the illumination engine1304 and to direct the EM radiation (or individual bandwidths thereof)along one or more predetermined optical paths.

For example, the illumination engine 1304 may emit the EM radiation intothe optical assembly 1306 along a common optical path that is shared byboth the first bandwidth and the second bandwidth. The optical assembly1306 may also include one or more optical components that are configuredto separate the first bandwidth from the second bandwidth (e.g., bycausing the first and second bandwidths to propagate along differentimage-generation and object-tracking optical paths, respectively).

In some instances, a user experience is dependent on the computingdevice 1300 accurately identifying characteristics of a physical object103 (a “real-world object”) or plane (such as the real-world floor) andthen generating the CG image in accordance with these identifiedcharacteristics. For example, suppose that the computing device 1300 isprogrammed to generate a user perception that a virtual gaming characteris running towards and ultimately jumping over a real-world structure.To achieve this user perception, the computing device 1300 might obtaindetailed data defining features of the real-world environment 112 aroundthe computing device 1300. In order to provide this functionality, theoptical system 1302 of the computing device 1300 might include a laserline projector and a differential imaging camera (both not shown in FIG.13 ) in some embodiments.

In some examples, the computing device 1300 utilizes an optical system1302 to generate a composite view (e.g., from a perspective of a userthat is wearing the computing device 1300) that includes both one ormore CG images and a view of at least a portion of the real-worldenvironment 112. For example, the optical system 1302 might utilizevarious technologies such as, for example, AR technologies to generatecomposite views that include CG images superimposed over a real-worldview. As such, the optical system 1302 might be configured to generateCG images via an optical assembly 1306 that includes a display panel1314.

In the illustrated example, the display panel includes separate righteye and left eye transparent display panels, labeled 1314R and 1314L,respectively. In some examples, the display panel 1314 includes a singletransparent display panel that is viewable with both eyes or a singletransparent display panel that is viewable by a single eye only.Therefore, it can be appreciated that the techniques described hereinmight be deployed within a single-eye device (e.g. the GOOGLE GLASS ARdevice) and within a dual-eye device (e.g. the MICROSOFT HOLOLENS ARdevice).

Light received from the real-world environment 112 passes through thesee-through display panel 1314 to the eye or eyes of the user. Graphicalcontent computed by an image-generation engine 1326 executing on theprocessing units 1320 and displayed by right-eye and left-eye displaypanels, if configured as see-through display panels, might be used tovisually augment or otherwise modify the real-world environment 112viewed by the user through the see-through display panels 1314. In thisconfiguration, the user is able to view virtual objects 104 that do notexist within the real-world environment 112 at the same time that theuser views physical objects 103 within the real-world environment 112.This creates an illusion or appearance that the virtual objects 104 arephysical objects 103 or physically present light-based effects locatedwithin the real-world environment 112.

In some examples, the display panel 1314 is a waveguide display thatincludes one or more diffractive optical elements (“DOEs”) forin-coupling incident light into the waveguide, expanding the incidentlight in one or more directions for exit pupil expansion, and/orout-coupling the incident light out of the waveguide (e.g., toward auser's eye). In some examples, the computing device 1300 furtherincludes an additional see-through optical component, shown in FIG. 13in the form of a transparent veil 1316 positioned between the real-worldenvironment 112 and the display panel 1314. It can be appreciated thatthe transparent veil 1316 might be included in the computing device 1300for purely aesthetic and/or protective purposes.

The computing device 1300 might further include various other components(not all of which are shown in FIG. 13 ), for example, front-facingcameras (e.g. red/green/blue (“RGB”), black & white (“B&W”), or infrared(“IR”) cameras), speakers, microphones, accelerometers, gyroscopes,magnetometers, temperature sensors, touch sensors, biometric sensors,other image sensors, energy-storage components (e.g. battery), acommunication facility, a global positioning system (“GPS”) a receiver,a laser line projector, a differential imaging camera, and, potentially,other types of sensors. Data obtained from one or more sensors 1308,some of which are identified above, can be utilized to determine theorientation, location, and movement of the computing device 1300. Asdiscussed above, data obtained from a differential imaging camera and alaser line projector, or other types of sensors, can also be utilized togenerate a 3D depth map of the surrounding real-world environment 112.

In the illustrated example, the computing device 1300 includes one ormore logic devices and one or more computer memory devices storinginstructions executable by the logic device(s) to implement thefunctionality disclosed herein. In particular, a controller 1318 caninclude one or more processing units 1320, one or more computer-readablemedia 1322 for storing an operating system 1324, and image-generationengine 1326 and a terrain-mapping engine 1328, and other programs (suchas a 3D depth map generation module configured to generate the depth mapdata (“mesh data”) in the manner disclosed herein), and data.

In some implementations, the computing device 1300 is configured toanalyze data obtained by the sensors 1308 to perform feature-basedtracking of an orientation of the computing device 1300. For example, ina scenario in which the object data includes an indication of astationary physical object 103 within the real-world environment 112(e.g., an engine), the computing device 1300 might monitor a position ofthe stationary object within a terrain-mapping field-of-view (“FOV”).Then, based on changes in the position of the stationary object withinthe terrain-mapping FOV and a depth of the stationary object from thecomputing device 1300, a terrain-mapping engine executing on theprocessing units 1320 AR might calculate changes in the orientation ofthe computing device 1300.

It can be appreciated that these feature-based tracking techniques mightbe used to monitor changes in the orientation of the computing device1300 for the purpose of monitoring an orientation of a user's head(e.g., under the presumption that the computing device 1300 is beingproperly worn by a user 102). The computed orientation of the computingdevice 1300 can be utilized in various ways, some of which have beendescribed above.

The processing unit(s) 1320, can represent, for example, a centralprocessing unit (“CPU”)-type processor, a graphics processing unit(“GPU”)-type processing unit, an FPGA, one or more digital signalprocessors (“DSPs”), or other hardware logic components that might, insome instances, be driven by a CPU. For example, and without limitation,illustrative types of hardware logic components that can be used includeASICs, Application-Specific Standard Products (“ASSPs”),System-on-a-Chip Systems (“SOCs”), Complex Programmable Logic Devices(“CPLDs”), etc. The controller 1318 can also include one or morecomputer-readable media 1322, such as the computer-readable mediadescribed above.

It is to be appreciated that conditional language used herein such as,among others, “can,” “could,” “might” or “may,” unless specificallystated otherwise, are understood within the context to present thatcertain examples include, while other examples do not include, certainfeatures, elements and/or steps. Thus, such conditional language is notgenerally intended to imply that certain features, elements and/or stepsare in any way required for one or more examples or that one or moreexamples necessarily include logic for deciding, with or without userinput or prompting, whether certain features, elements and/or steps areincluded or are to be performed in any particular example. Conjunctivelanguage such as the phrase “at least one of X, Y or Z,” unlessspecifically stated otherwise, is to be understood to present that anitem, term, etc. may be either X, Y, or Z, or a combination thereof.

It should also be appreciated that many variations and modifications maybe made to the above-described examples, the elements of which are to beunderstood as being among other acceptable examples. All suchmodifications and variations are intended to be included herein withinthe scope of this disclosure and protected by the following claims.

In closing, although the various configurations have been described inlanguage specific to structural features and/or methodological acts, itis to be understood that the subject matter defined in the appendedrepresentations is not necessarily limited to the specific features oracts described. Rather, the specific features and acts are disclosed asexample forms of implementing the claimed subject matter.

What is claimed is:
 1. A method for facilitating communication between a plurality of computing devices and a virtual agent, comprising: receiving, at a computing device of the plurality of computing devices, object data defining at least one of a real-world object or virtual object of interest displayed in a user interface in communication with the computing device; generating agent data defining at least one instance of a virtual agent associated with the at least one real-world object or virtual object of interest, the at least one agent instance having a database for individual objects of interest that are identified by an analysis of the object data; receiving communication data associated with at least one object of interest from at least one computing device of the plurality of computing devices; storing at least one attribute associated with the at least one object of interest from the communication data at the database of the at least one virtual agent instance, wherein the at least one attribute associated with the at least one object of interest is configured for access by multiple communication sessions; receiving a query interpreted from communication data from a computing device of the plurality of computing devices regarding one or more objects of interest; and causing the computing device to retrieve the at least one attribute associated with the one or more objects of interest from the database for display of the at least one attribute of the one or more objects of interest with a display of the communication data in response to the query.
 2. The method of claim 1, wherein the at least one instance of the virtual agent is communicated to a management agent at a server to be persistently stored for access by the multiple communication sessions.
 3. The method of claim 1, wherein the plurality of computing devices and the at least one instance of the virtual agent are in communication during a communication session, wherein an individual communication session of the multiple communication sessions defines parameters of a meeting.
 4. The method of claim 1, wherein the data defining the at least one instance of the virtual agent is configured to persist following conclusion of a communication session for access across the multiple communication sessions.
 5. The method of claim 1 wherein the instance of the virtual agent and associated database are generated during a first communication session, further comprising: determining that a second communication session has started; and in response to determining that a second communication session has started, communicating agent data defining the agent instance and the database to one or more computing devices associated with the second communication session.
 6. The method of claim 1, wherein the at least one attribute is configured to persist following conclusion of a first communication session for access across the multiple communication sessions, and wherein the at least one attribute associated with the at least one object of interest is from the first communication session, and wherein the query is from a second communication session.
 7. The method of claim 1, further comprising: monitoring user interaction data of a plurality of users from one or more input devices for identifying the at least one attribute, wherein the at least one attribute comprises at least one of a parameter, a keyword, an image, a model, or a description of the one or more objects of interest.
 8. The method of claim 1, wherein the objects of interest are identified by: determining a level of interaction of input signals received from input devices with respect to the object data defining the at least one real-world object or virtual object; determining that the level of interaction exceeds a threshold; and in response to determining that the level of interaction exceeds the threshold, determining the at least one object as an object of interest.
 9. A computing system comprising: one or more data processing units; and a computer readable device having encoded thereon computer-executable instructions that cause the one or more data processing units to: receive, at a computing device of the plurality of computing devices, object data defining at least one of a real-world object or virtual object of interest displayed in a user interface in communication with the computing device; generate agent data defining at least one instance of a virtual agent associated with the at least one real-world object or virtual object of interest, the at least one agent instance having a database for individual objects of interest that are identified by an analysis of the object data; receive communication data associated with at least one object of interest from at least one computing device of the plurality of computing devices; store at least one attribute associated with the at least one object of interest from the communication data at the database of the at least one virtual agent instance, wherein the at least one attribute associated with the at least one object of interest is configured for access by multiple communication sessions; receive a query interpreted from communication data from a computing device of the plurality of computing devices regarding one or more objects of interest; and cause the computing device to retrieve the at least one attribute associated with the one or more objects of interest from the database for display of the at least one attribute of the one or more objects of interest with a display of the communication data in response to the query.
 10. The system of claim 9, wherein the at least one instance of the virtual agent is communicated to a management agent at a server to be persistently stored for access by the multiple communication sessions.
 11. The system of claim 9, wherein the plurality of computing devices and the at least one instance of the virtual agent are in communication during a communication session, wherein an individual communication session of the multiple communication sessions defines parameters of a meeting.
 12. The system of claim 9, wherein the data defining the at least one instance of the virtual agent is configured to persist following conclusion of a communication session for access across the multiple communication sessions.
 13. The system of claim 9 wherein the instance of the virtual agent and associated database are generated during a first communication session, wherein the computer-executable instructions further cause the one or more data processing units to: determine that a second communication session has started; and in response to determining that a second communication session has started, communicate agent data defining the agent instance and the database to one or more computing devices associated with the second communication session.
 14. The system of claim 9, wherein the at least one attribute is configured to persist following conclusion of a first communication session for access across the multiple communication sessions, and Wherein the at least one attribute associated with the at least one object of interest is from the first communication session, and wherein the query is from a second communication session.
 15. A computer storage medium device having encoded thereon computer-executable instructions that cause a system to: receive, at a computing device of the plurality of computing devices, object data defining at least one of a real-world object or virtual object of interest displayed in a user interface in communication with the computing device; generate agent data defining at least one instance of a virtual agent associated with the at least one real-world object or virtual object of interest, the at least one agent instance having a database for individual objects of interest that are identified by an analysis of the object data; receive communication data associated with at least one object of interest from at least one computing device of the plurality of computing devices; store at least one attribute associated with the at least one object of interest from the communication data at the database of the at least one virtual agent instance, wherein the at least one attribute associated with the at least one object of interest is configured for access by multiple communication sessions; receive a query interpreted from communication data from a computing device of the plurality of computing devices regarding one or more objects of interest; and cause the computing device to retrieve the at least one attribute associated with the one or more objects of interest from the database for display of the at least one attribute of the one or more objects of interest with a display of the communication data in response to the query.
 16. The computer storage device of claim 15, wherein the at least one instance of the virtual agent is communicated to a management agent at a server to be persistently stored for access by the multiple communication sessions.
 17. The computer storage device of claim 15, wherein the plurality of computing devices and the at least one instance of the virtual agent are in communication during a communication session, wherein an individual communication session of the multiple communication sessions defines parameters of a meeting.
 18. The computer storage device of claim 15, wherein the data defining the at least one instance of the virtual agent is configured to persist following conclusion of a communication session for access across the multiple communication sessions.
 19. The computer storage device of claim 15, wherein the instance of the virtual agent and associated database are generated during a first communication session, wherein the computer-executable instructions further cause the system to: determine that a second communication session has started; and in response to determining that a second communication session has started, communicate agent data defining the agent instance and the database to one or more computing devices associated with the second communication session.
 20. The computer storage device of claim 15, wherein the at least one attribute is configured to persist following conclusion of a first communication session for access across the multiple communication sessions, and wherein the at least one attribute associated with the at least one object of interest is from the first communication session, and wherein the query is from a second communication session. 